DORY: Automatic End-to-End Deployment of Real-World DNNs on Low-Cost IoT MCUs

The deployment of Deep Neural Networks (DNNs) on end-nodes at the extreme edge Internet-of-Things is a critical enabler to support pervasive Learning-enhanced applications. Low-Cost MCU-based have limited on-chip memory and often replace caches with scratchpads, reduce area overheads increase energy efficiency - requiring explicit DMA-based transfers between different levels hierarchy. Mapping modern DNNs these systems requires aggressive topology-dependent tiling double-buffering. In this work, we propose DORY (Deployment Oriented memoRY) an automatic tool deploy low cost MCUs typically less than 1MB SRAM memory. abstracts as Constraint Programming (CP) problem: it maximizes L1 utilization under topological constraints imposed by each DNN layer. Then, generates ANSI C code orchestrate off- computation phases. Furthermore, maximize speed, augments CP formulation heuristics promoting performance-effective tile sizes. As case study for DORY, target GreenWaves Technologies GAP8, one most advanced parallel ultra-low power MCU-class devices market. On device, achieves up 2.5× better MAC/cycle proprietary software solution 18.1× state-of-the-art result STM32-H743 MCU single layers. Using our tool, GAP-8 can perform end-to-end inference 1.0-MobileNet-128 network consuming just 63 pJ/MAC average @ 4.3 fps 15.4× STM32-H743. We release all developments framework, optimized backend kernels, related open-source software.